Recognize, understand and distribute voice Commands

The robot has skills such as "dancing", "singing", and "navigating". The user can trigger the corresponding skills by speaking to the robot (sending voice commands). For example, if the user says "Dance for me", the robot will start dancing. From the user sending an instruction to the robot carrying out the instruction, the robot's working steps are as follows:

1. Recognition: The robot notices that a user is talking to it and recognizes what the user is saying
2. Understand: To understand the meaning of the user’s words and translate them into the corresponding instructions
3. Distributing instructions: To distribute instructions through a certain mechanism.
4. Execute the instruction.

The above steps are implemented through the following code:

Preparations

Preparations include setting up a development environment, creating a project, adding library dependencies, etc. For details, see build your first app.

Recognition

To enable the robot's "listening" skills, you can use the speech recognition or speech wake-up technology. To use speech recognition or speech wake-up technology, you need to access the API provided by the SpeechManager proxy object through the speech service. For creating the SpeechManager object, see speech service.

The API used in the identification process comes from the speech service. The detailed information such as parameters, interfaces, and proprietary names that appear in this implementation process can be found through the speech service.

To use speech recognition technology to enable the robot's "listening", you can use the following code.

speechManager.recognize(new RecognitionOption.Builder(
    RecognitionOption.MODE_CONTINUOUS).build())
    .progress(new ProgressCallback<RecognitionProgress>() {
        @Override
        public void onProgress(RecognitionProgress recognitionProgress) {
            if (recognitionProgress.getProgress()
                .equals(RecognitionProgress.PROGRESS_RECOGNITION_TEXT_RESULT)) {
                String asr = recognitionProgress.getTextResult();
                understand(asr); // execute the understanding operation
            }
        }
    })
    .fail(new FailCallback<RecognitionException>() {
        @Override
        public void onFail(RecognitionException e) {
            // if the listening goes wrong, it will run here
        }
    });

To use speech wake-up technology to enable the robot's "listening" skill, you can use the following code.

speechManager.registerWakeUpListener(new WakeUpListener() {
    @Override
    public void onWakingUp(WakeUp wakeUp) {
        // After the robot perceives to be awakened, it can perform the following tasks: adjust posture (facing the user), enable recognition
        // enable identification
        speechManager.recognize(new RecognitionOption.Builder(
            RecognitionOption.MODE_SINGLE).build())
            .done(new DoneCallback<RecognitionResult>() {
                @Override
                public void onDone(RecognitionResult recognitionResult) {
                    if (!recognitionResult.withUnderstandingResult()) {
                        String inputText = recognitionResult.getText();
                        understand(inputText); // execute the understanding operation
                    }
                }
            })
            .fail(new FailCallback<RecognitionException>() {
                @Override
                public void onFail(RecognitionException e) {
                    // if the listening goes wrong, it will run here
                }
            });
    }
});

If the robot's "listening" skill is turned off, the user needs to use a non-speech method (click the button) to trigger it. If using wake-up technology, the user can turn on the listening skill through wake-up words.

Understanding

If the robot wants to understand what the user says, it can use natural language processing technology. The code is presented below.

public Promise<UnderstandingResult, UnderstandingException> understand(String inputText) {
    return speechManager.understand(inputText)
        .done(new DoneCallback<UnderstandingResult>() {
            @Override
            public void onDone(UnderstandingResult understandingResult) {
                String action = understandingResult.getIntent().getAction();
                dispatchDirective(action); // execute the operation of distributing orders
            }
        }).fail(new FailCallback<UnderstandingException>() {
            @Override
            public void onFail(UnderstandingException e) {
                // if the listening goes wrong, it will run here
            }
        });
}

Distribute instructions

public Promise<Void, DispatchException> dispatchDirective(String action) {
    return Robot.globalContext().dispatchDirective(action).done(new DoneCallback<Void>() {
                @Override
                public void onDone(Void aVoid) {
                    // if the instruction distribution is completed, it will run here
                }
            }).fail(new FailCallback<DispatchException>() {
                @Override
                public void onFail(DispatchException e) {
                    // if the instruction distribution goes wrong, it will run here
                }
            });
}

Execute instructions

The application that executes an instruction is called a skill application. It is mainly used to receive an instruction. After receiving the instruction, it can be executed by calling the corresponding system service. The skill application and the instruction distribution application can be the same application or different applications. If they are different applications, you need to create them. To create applications, please refer to build your first app. To realize the skills, please refer to skills.